# Large-scale Pre-training

The Gemma-2-Llama-Swallow series is built through continued pre-training of the gemma-2 model, significantly enhancing Japanese language processing capabilities while retaining original English proficiency.

LHM is a feed-forward model capable of reconstructing an animatable 3D human body from a single image within seconds.

SimCSE (Supervised Version) is a Chinese sentence similarity calculation model based on supervised learning, optimizing sentence embeddings through contrastive learning.

MERT-v1-330M is an advanced music understanding model trained based on the MLM paradigm, with 330M parameters, supporting a 24K Hz audio sampling rate and 75 Hz feature rate, suitable for various music information retrieval tasks.

This is a Hindi automatic speech recognition (ASR) model fine-tuned based on OpenAI Whisper Large V2, trained on the Common Voice 11.0 dataset with a word error rate (WER) of 10.05%.

A Japanese automatic speech recognition model fine-tuned using the Common Voice 7.0 (Japanese) dataset, based on the microsoft/unispeech-large-1500h-cv model

A Chinese sentiment analysis model based on the MegatronBert architecture, fine-tuned on multiple sentiment analysis tasks

A fine-tuned version based on the Chinese RoBERTa-wwm-ext-large model on multiple natural language inference task datasets

DeBERTa is an improved BERT model that enhances performance through a disentangled attention mechanism and an enhanced masked decoder, surpassing BERT and RoBERTa in multiple natural language understanding tasks.

An English fine-tuned speech recognition model based on facebook/wav2vec2-large, using the Common Voice dataset, supporting 16kHz sampled audio input.

Chinese pre-trained model based on the BigBird architecture, employing Whole Word Masking (WWM) strategy, supporting a 4096-length context window

This is a French automatic speech recognition model fine-tuned on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - FR dataset based on facebook/wav2vec2-xls-r-1b.

DeBERTa is an enhanced BERT decoding model based on the disentangled attention mechanism, surpassing BERT and RoBERTa in multiple natural language understanding tasks by improving the attention mechanism and masked decoder.